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ABSTRACT 

The Wi 1 1 iamsburg/ James City County (W.iC) , Virginia 
School System employs a wide variety of professional support 
personnel (PSP) . Little has been done to properly evaluate and 
monitor PSP or programs with which they are involved as the focus is 
on classroom teacher evaluation. PSP are reluctant to be evaluated by 
persons who have little knowledge of PSP professional roles and 
responsibilities, and who use methods with little relevance for PSP 
positions. The purpose of this project is to provide a framework for 
evaluating five full-time and one part-time WJC enrichmenl 
specialists based on the method outlined in the book, "Evaluating 
Professional Support Personnel in Education" (James H. Stronge and 
Virginia M. Helm). Steps identified to reach program evaluation goals 
were: (1) Identify System Needs; (2) Identify Duties; (3) Select 
Performance Indicators; (4) Set Performance Standards; (5) Document 
Job Performance; (6) Evaluate Performance; and (7) Improve/Maintain 
Professional Services. Appendices, making up one-third of the 
document, detail specific WJC program evaluation components such as 
an evaluation time line, schoolwide enrichment program review form, 
job areas and duties of an enrichment resource specialist, a 
checklist for developing appropriate standards, and an artifact 
analysis form. (JBJ) 
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introduction 



The Williamsburg/James City County (WJC) School System, in recognition of the diverse needs of its 
student population and state and federal legislation, employs a wide variety of professional support personnel 
(PSP). However, little has been done to properly evaluate and monitor these employees or the programs with 
which they are involved. Inasmuch, the WJC Schools are in line with most school systems nationwide as ". . 
.few would dispute the suggestion that PSP evaluations in the past generally have been rare, inadequate, or 
both " when conducted at all" (Stronge & Helm, 1991). The focus instead seems to be on classroom teachers 
with observation being the primary method for documenting teacher competencies (Stronge & Helm, 1992). 

With PSP employees firmly entrenched as permanent and necessary fixtures within the schools, it is 
reasonable that they should expect the development and use of evaluation procedures and tools specifically 
designed to honor the job responsibilities, performance indicators, and best modes of data collection for their 
respective positions. No professional shies away from evaluation, especially when it is designed to enhance 
professional growth and the quality of one's work performance as they relate to the attainment of individual 
professional, program, and system goals. On the other hand, it is easy to understand why PSP are reluctant 
"to be evaluated by someone whom they regard as knowing little it anything about their professional roles and 
responsibilities" (Stronge & Helm, 1991). This reluctance is compounded when the tools for and modes of 
data collection have little relationship to the duties of the employee being evaluated. 

Such is the case for the five full-time and one part-time Schoolwide Enrichment Specialists in the WJC 
Schools These PSP are evaluated (or at least observed - evaluation and observation seem to be 
synonymous in this district) using a checklist developed for classroom teachers. Very little attention is given 
the actual )ob responsibilities of these specialists. As a matter of fact, in order to be observed, a specialist must 
arrange to be engaged with a group of students in some kind of activity that can be equated with a typical 
classroom setting' 

Statement of the Problem 

"High quality programs do not happen by accident. In addition to careful planning and a commitment 
to carrying out specific responsibilities on the parts of all persons involved, a system for evaluation and 
monitonng must be built into the overall programming model" (Renzulli & Reis, 1985). Likewise the personnel 
who serve within each program must be evaluated to determine if they are performing their jobs competently 
and it the jobs accurately reflect both the program goals and the organization's mission. With the alignment of 
individual objectives, program goals, and the organization's mission, the evaluation of individual staff 
performance will provide significant information for evaluating the effectiveness of the organization's programs 
and services, and the progress toward achieving the institutional goals and missions (Stronge & Helm, 1991). 
This voluntary effort may lead to program inprovement and increased public support and approval (Knapper, 
1978). 

In the WJC Schools, enrichment resource specialists have replaced the traditional G/T teacher. These 
specialists are school-level administrators of a multi-faceted program who need and deserve an evaluation 
model that assesses their strengths and weaknesses in relation to the program goals and provides feedback 
for continued professional growth and development. The purpose of this project is to provide a framework for 
evaluating the WJC ennchment specialists based on the method outlined in the book. Evaluating Professional 
Support Personnel in Education by James H. Stronge and Virginia M. Helm (1991 ). The VISIONS/ 
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Schoolwide Enrichment Program must take the initiative to create and institute an effective evaluation 
procedure for its staff members and ultimately for the program itsetf . By taking this initiative, the VISIONS 
personnel may become true agents of chsinge as other PSP groups adopt this model and tailor it to match 
their own program goals and personnel duties. 

Review of Literature 
There is a dearth of literature concerning the evaluation of gifted program personnel. Much 
can be found about program evaluation, but the "evaluaton' of the staff members seems to be limited to 
listings of teacher competencies, characteristics, and behaviors. This is probably due to the fact that in most 
cases the teachers of the gifted are regular classroom teachers who have identified gifted students in their 
classrooms or are teachers assigned to "pull-out" gifted students for special instruction and experiences. It is 
these characteristics and behaviors which make it likely that a teacher will differentiate the curriculum content, 
processes, and products for the gifted students. However, these lists have no real relationship to program 
goals, personnel duties, or performance indicators; therefore, they are ill-suited to be used as evaluation 
standards. 

There is no shortage of literature on the evaluation of professional educators (teachers); yet, there is 
Irttle dedicated solely to the unique concerns of evaluating professional support personnel. One can easily 
synthesize, from the literature geared toward the evaluation of teachers, a list of common purposes and 
principles related to the assessment and evaluation of any professional. Sources such as Retxjre (1987), 
McGreal (1983), and Stiggins & Duke (1988) contribute to this list of commonalities 

1 . To foster the self-development of each employee; 

2 To Identify a variety of tasks that an employee is capable of performing; 

3. To identify staff development needs; 

4 To improve performance: 

5 To determine whether or not an employee should be retained, 

6 To determine the potential for promotion and/or salary increase. 

7 To protect students and school employees from incompetence. 

8 To assess the system's employee selection process. 

9 To provide a basis for career planning and profe.'-.sional development. 

1 0 To provide individuals and institutions with indicators regarding expectations and areas of 
res^x)nsibiiity, authority, and discretion, and, 

1 1 To provide a legal and ethical framework of operation 

Consider in addition to this list of reasons for evaluation this definition for evaluation developed by 
Stronge and Helm (1991 ) which alludes to the purpose of evaluation as they see it "Evaluation is a process of 
determining the degree to which an employee s or a program's objectives have been achieved in order to 
improve continually the educational institution's ability to accomplish its mission" (p 25). 

While there isn1 a great deal of specific literature on the evaluation of PSP, there are eight critical 
attributes of effective evaluation systems identified by David T. Conley and cited by Stronge and Helm in their 
book (1991, p. 12-14). These attributes address the question, "What are some of the key elements that will 
help an evaluation system address the often conflicting needs of organizational accountability and individual 
growth"?" They include: 

1 . All participants accept the validity of the [evaluation] system. 

2. All participants thoroughly understand the mechanics of the system. 



3 . Evaluatees know that the periormance criteria have a clear, consistent 
rationale. 

4. Evaluators are properly trained in the procedural and substantive use of the 
system. 

5. Levels of evaluation are employed, each with a different goal. 

6. The evaluation distinguishes between the formative and summative 
dimensions. 

7. A variety of evaluation methods are used. 

8. Evaluation is a district priority. 

In their article, 'Evaluating Educational Support Personnel; A Conceptual and Legal Framework", 
Stronge and Helm (1990) listed four basic attributes for sound evaluation as defined by the Joint Committee 
on Standards for Educational Evaluation in 1988. These four attributes include: propriety standards, utility 
standards, feasibility standards, and accuracy standards. 

An investigation of the literature on evaluation by Beckhouse, Cobb, Moody, Morgan, and Price 
(1993) revealed factors critical to the evaluation of educational personnel such as the need for a positive 
focus built on cooperative attitudes, trust, and clear communication. They also found that exemplary 
evaluation systems must consider institutional needs, goal setting, measurable performance indicators, 
constructive feedback, and an understanding of the differences between formative and summative appraisal. 
Also critical to appropriate evaluation of professional support personnel is the need for a valid, reliable, and 
useful measure that can be adapted to categories, positions, and individuals (Stronge & Helrh, 1991). The 
need for using a variety of sources of data, for comprehensive staff development, and for cooperative input by 
building-based and program administrators or supervisors is cited by McGreal (1983). Sensitivity to the 
particularistic nature of the educator, position, and educational context is critical (Popham, 1988). 

Any evaluation model that is to be used successfully for PSP must resist the temptation to draw a 
parallel between task observation of professional support personnel and classroom observation of regular 
teachers (Poster, 1991 ). It is imperative that support personnel be appraised by a person(s) who has 
management responsibility for him or her as well as considerable knowledge atxjut program goals pertinent to 
the PSP's role. For many PSP evaluations, this will require interplay between a building administrator and a 
program coordinator. Appraisal tools and methods must include task observation and artifacts collection and 
should take into consideration additional [data sources] such as: peer evaluations, student evaluations, self- 
evaluations, and student performance. Adaptations must be based on job specifications of the staff memtjer, 
system needs, and cooperatively developed periormance indicators and standards (Poster, 1931 ; Stronge & 
Helm, 1992) 



Figure 1 atxjut here 



The seven-step model proposed by Stronge and Helm (1991) in their text. Evaluating 
Professional Support Personnel in Education is an excellent model which can be used by the WJC 
Schools to evaluate all PSP including the Schoolwide Enrichment Specialists who staff the VISIONS Progrxim 
(see Figure 1). Stronge and Helm have sought to integrate all the characteristics, attributes, and observations 
atxjut effective evaluations which have been mentioned thus far. Their model is congruent with standards 
developed by the Joint Committee for Educational Evaluation and includes the following key elements: (1) an 
emphasis on the four basic attributes of sound evaluation propriety, utility, feasibility, and accuracy; (2) 



application of systems theory; (3) emphasis on communication; (4) reliance on muttitaceted data collection; 
and (5) emphasis on improvement of job performance in relation to the educational organization's goals 
Though the model put forth by Stronge and Helm serves as a framework, the development of the actual 
evaluation process (and any instruments) is still meant to be an individualized design that directly corresponds 
to the position being considered, the program's goals, and the system's mission. However, the seven-steps 
are so thoroughly descrit>ed that this text literally functioned as a handtxxik for the proposed enrichment 
specialist's evaluation found on the following pages. 

Organizational Goals and Personnel Evaluation 

The mission statement of the WJC Schools and the goal statement of the Schoolwide Enrichment 
Program (VISIONS) a-'e somewhat ambiguous and too loosely defined to serve as the evaluation standard for 
the Schoolwide Enrichment Program or the professional support personnel (enrichment specialists) who staff 
it. However, the mission statement provides a clear foundation when it is used as a guideline for developing 
more specific school/program goals, school/program expectations, and job responsibilities from which 
performance indicators can t>e created. 

The WJC mission statement has within its text key phrases which should serve as the impetus for the 
goal statements and improvement plans established by each school and program within the system. Some of 
the phrases can lead to a rather broad interpretation which, some would argue, render them meaningless and 
futile (Castetter, 1981 ; Patton, 1986). However, with provisions for open, honest, and constant 
communication and feedback, they should serve as guidelines for the individual schools and the programs 
within the schools, thus allowing for more site-based management and shared decision making - two 
important trends in education today. 

For instance, the phrase "...through partnership with home and community," is in keeping with the 
new paradigms of school reform which dictate that schools must t»ecome more integral parts of the community 
and that communities must embrace schools and help in the training of young people for their roles in society. 
Each school within the WJC district, being a system within a system, must find ways to address this guideline in 
Its school improvement plan with specific, measurable oojectives and workable and realistic strategies. By the 
same token, special programs, such as the Schoolwide Enrichment Program, must also determine ways to 
honor this guideline as it establishes program goals and expectations and personnel responsibilities. 

Therefore, one way to begin to institute an evaluation system for the professional support personnel 
who work as schoolwide enrichment specialists is to closely examine the context of the WJC mission 
statement to be sure that the Schoolwide Enrichment Program s goal statement is in keeping with the 
guidelines put forth (see Figure 2) 



Figure 2 about here 



If the WJC mission statement is the guideline, then all other goal statements (those of the individual 

schools and programs) have the burden of being more cleariy defined and specifically stated. For this reason, 

the existing goal statement of the Schoolwide Enrichment Program is too ambiguous to be useful in the 

personnel (or program) evaluation process; 

Goal Statement of the Schoolwide Enrichment Program (As of 8/93) 

The goal of the Schoolwide Enrichment Program in the Williamsburg/James City County 
Public School System is to nurture students' strengths, talents, and sustained interests by 
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providing opportunities for students to become creative producers and life-long learners. To 
achieve this goal, we seek to develop gifted behaviors in students through three types of 
enrichment activities offered within each school. 

In order to make the program goals dear to each Enrichment Specialist whose job it is to achieve these 
goals, the expectations must be clearly delineated. This may entail having a goal statement that is kjnger than 
one paragraph. The dilemma seems to be one of audience. So often goal statements are shared with parent 
groups and community members as a matter of put>lic relations. The idea behind a simplistic statement then 
being that these people lack expertise in the field of education and would be unable to fathom a lengthier 
description of what we are about. Or worse yet, there is perhaps a fear that we don't really want them to know 
what we are about and be able, as taxpayers, to hold us accountable. At this point, it may be necessary to 
pnoritize the many, different uses for goal statements. Rather than using the goal statement itself as a means 
of communicating with the public, it would be better to use these goals as the driving force behind the 
organization (or system) reaching its desired state of being and let the results of this meaningful effort speak to 
the public for themselves. 

With this in mind and in keeping with the Mission Statement of the Williamsburg/James Qty County 
School Division, the goals of the Schoolwide Enrichment Program can be pulled directly from the list of 
interpretations on Figure 2. One might even argue that formatting goals in a list facilitates the subsequent 
steps in evaluating professional support personnel -- Step 2: Identifying Duties. 




The idea that schools and programs are systems within a greater system has already been mentioned. 
It should also be noted that often this program subsystem is constantly interacting with one or more 
subsystems schools, levels (i.e., elementary, middle, and high school), and/or other programs. This can 
have unique implications for needs assessment and planning For instance, the WJC VISIONS Program 
coordinator at the distnct level would have to conduct the needs assessment for the program as a whole. 
Whereas, the enrichment specialists who service the three middle schools would work together with the 
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district coordinator to design and conduct a needs assessment for the middle school level of the program. 
And, of course, each enrichment specialist would work to ascertain the needs within the school(s) he or she 
services. Rather than beginning the program review process at the district level, the review process should 
begin at each school, move through the level review, and finally the program should be reviewed at the district 
level. By moving through the review process in this manner, the pnmary constituents - the students, and the 
front lines of operations - the schools remain in the forefront. 

For this purpose, a program review form can be used first by specialists in the individual schools, then 
by teams of specialists representing the three levels, and finally by the distnct level coordinator to compile an 
adequate description of the existing program. The Program Review Form in the appendix should serve as an 
aid to reviewing the current program. 

Likewise, the assessment of future needs must be done using this three tiered approach in order to 
make the decisions to keep, modify, delete, expand, or add services and delivery modes which are necessary . 
to mold the greater system and its subsystem components toward their vision for the future (Stronge and 
Helm, 1991 ). The infonnation gleaned Irom the review of the district mission statement, the existing program, 
and the program goals can be used to create needs assessment documents designed to reveal the specific 
future needs of the program as it is nonifested in each school, at each level, and district-wide. 




Stronge and Helm (1991) stated that '[fjor congruence to exist between the organization's goals and 
performance, the assessed needs of the organization must serve to clarify the expectations of all programs 
(including professional support programs), which in turn define the staff job responsibilities [duties] associated 
with a given program. If a school district or any educational organization is to actuate an effective and efficient 
state of being, it must achieve alignrnent stretching from the organization's broad goals to the front line of 
personnel service delivery" (p 98. 99) 

Just as It IS absolutely essential that the goals of the organization and program be clesirly defined, it is 
imperative that clear and absolute )ob responsibilities and expectations be delineated for employees. 
Somewhere along the line tne assumption has tieen made that once a person is hired as a practicing 
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professbnal, he or s*ie no tonger requires direction and guidance. It is as it the act of hiring someone 
automatically imbues him or her with a complete understanding of the job expectations (Redfem, 1980). 

This is especially true in non-traditional roles because they are, well..., non-traditional. Professional 
support personnel oiten do not have the advantage of being surrounded by coworkers who have the same 
roles within the organization. There are very few models for these people. Likewise, they are often the 
professional "in charge" of a fairly new or unique program or service that does not have a lengthy track record 
and clearly understood expectations. Considering these elements of chance, leaving the job expectations 
undefined will certainly cause professional bum-out and breakdown if not program failure. 

The role of the enrichment resource specialist is unique in that it falls into several of the support 
personnel categories listed by Stronge and Helm (1991) in their book, Evaluating Professional Support 
Personnel in Education. Elements of the duties of an enrichment resource specialist would justify 
classification of the position as all of the following: pupil personnel services, instructional support services, and 
academic/ program development services. Because of the nature of this multi-faceted role, it is especially 
imperative that a comprehensive job description or duties list be provided the personnel who work as 
enrichment resource specialists. The list of duties provided in the appendix is a comprehensive list for WJC 
schoolwide enrichment specialists based on the current prograim description and goals. 

The lengthy but explicit list of the duties would seem to indicate that a future needs assessment and 
program review may reveal the need for eliminating aspects of the program or bringing additional personnel on 
txDard. As a matter of fact, the numt>er of duties, while there is overlap in some job areas, would make it nearly 
impossible to run a program tjecause .most of the time the specialist would be involved in some aspect of 
evaluation. However, this list does reflect the current expectations of these specialists. 



While duties do indeed describe the functions of an employee as they relate to evaluation, they do 
not lend themselves to direct measurement. It is almost impossible to readily evaluate the essence of the job 
as captured in the list of duties without using highly subjective and speculative judgements Therefore, it is 
necessary to translate the implications of these duties into performance indicators (using the same rationale as 
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Step 3: Select 
Performance Indicators 



11 



ERIC 



performance objectives) that are both measurable and indicative of the responsibility in question (Stronge and 
Helm, 1991). Using appropriate performance indicators has many benefits in the evaluation process 
They allow for direct, objective measurement. 

They allow appraisal procedures to ascertain the extent to which an individual exhibits 
performance commensurate with the objectives. 

• The use of performance indicators builds collegiality and collaboration as evaluators and 
evaluatees work together to define them. 

Performance indicators enhance the interactive formative and summative aspects of personnel 
evaluation. 

They generate more clearly defined feedback for the employee at conferencing. 

• They provide insight into program needs or staff development needs, or any potentially 
necessary modifications to the organization's goals. 

Stronge and Helm (1 591) also listed additional key points about performance indicators that can serve 
as guidelines when creating such indicators no matter what the position, the area, or the specific duty. 

Performance indicators are the obsen/able activities that are related to the performance of duties 

■ • Performance indicators should be selected for inclusion in the evaluation based on their 
reresentativeness of the domain of a particular duty. 

Performance indicators are activities whose performance can be objectively documented and 
measured. 

Because performance indicators can be documented, they can be measured in a quantitative 
fashion; ti^ey are the objective basis tor evaluation. 

As an objective basis for evaluation, performance indicators are used as a sample of behavior for 
assessing performances of duties. 

Performance indicators are intended merely as a sample and not as a full set of behaviors related 
to duties. 

Add to this list the idea that performance indicators should adequately specify worthy and realistic objectives 

(Harris. Mclntyre. Littleton, & Long, •<985. Bolton, 1980). 

To be sure there are any number of t>ehaviors associated with the many, different duties of any 

position. However, there are selection critena which, when employed will help quarantee that the best 

possible indicators are being used These critena address the concepts of representativeness, accuracy, and 

importance (Stronge and Helm, 1 991 ) These concepts can be defined as follows: 

representativeness = Performance indicators represent the body of behaviors associated with a 
given duty. They are not intended to be comprehensive and all encompassing Care should be 
taken, by both the evaluator and the evaluatee, to select indicators which will encourage a 
mutually satisfactory evaluation process. 

accuracy = Primary indicators (e g . time on task, preparation, continued education) versus 
secondary indicators are recommended as they are more accurate measures of performance 
cnteria. 
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importance = Three factors should be taken into consideration when determining the level of 
importance of performance indicators: health and safety, educational significance, and 
cumulative effect of staff behaviors. Any one of these factors or a combination of these factors 
may elevate a perfonnance indicator to position of real importance. 

It is very important that "all users of the accountability system ... be represented in designing it" 
(Krumbottz, 1974). All staff members, coordinators, pnncipals, etc. must work together to generate 
appropriate performance indicators while keeping in mind the situational variables and unique needs that exist 
in the given organizational setting (Stronge and Helm, 1991). A 'menu" of performance indicators has been 
created by the VISIONS staff (see Appendix). Building administrators, advisory council members, and others 
associated witht the program will be given the opportunity to respond to these indicators and to submit 
additional indicators. As a result a thorough list of indicators will be available, so this step need not be 
recreated each year. Instead, newly created indicators will simply be added to the existing list. A thorough 
review of this step need only occur on the regular program evaluation cycle every three to five years. 

At this point in the development of an evaluaton process for enrichment resource specialists, it is 
necessary to address the role of two key people involved in the evaluation process. Enrichment resource 
specialists in William sburg/James City County are in the unique position of being central office employees who 
are assigned to individual schools. This means that each specialists has a building administrator (or two - if 
assigned to more than one school) and a central office coordinator who both serve in supervisory capacities 
for the specialist. Interaction with and input from both of these individuals as evaluators is necessary. The 
building administrator has a keen awareness of the goals and needs of the school, whereas, the central office 
coordinator has more expertise in the role of enrichment programs in schools and the needs of high ability 
learners (the primary clients of the program) This strengthens the argument for a multi-faceted evaluation 
process for these professional support personnel. 




Set Performance Standards 

Pertiaps one of the most time consuming yet vital aspecis of the evaluation process is the creation of 
standards. By definition, "[sjtandards . are statements of predetermined levels of acceptable results. 



understood in light of the context and viewed with attention to the processes which were utilized to achieve 
those results" (Stronge & Helm, 1991 , p. 148). The standards that will be used to evaluate the level of 
performance of an individual must be set prior to documenting job performance. Both the evaluator and 
evaluatee should t»e involved in creating the standards and great care should be taken to assess the 
appropriateness of the standards on the basis of several important criteria. These criteria have been 
incorporated into a form (see Appendix) that will help novices consider each criterion for every standard that is 
written. Administrators (evaluators) may have more training or experience in the rudiments of writing 
performance standards but will find the form and collaboration with the specialist most helpful in the evaluation 
process. 

As was mentioned t>efore, all users of the accountability system must be represented in its design. 
That is the enrichment specialist (PSP), the program coordinator, and the building administrator should all be 
involved in creating the standards. In order to streamline this step, the enrichment specialist should develop a 
standard for each performance indicator that honors the criteria for creating standards or should choose one 
from the "menu" of standards (currently t>eing developed by the VISIONS staff). A Hst of all the standards 
should then be submitted to tx>th the program coordinator and the building administrator about two weeks 
pnor to a meeting (which will ideally t>e held before the school year tjegins -- see the time line in the appendix). 
The program coordinator and building administrator should review the list with the critena at hand and t>e 
prepared to make comments and/or suggest changes. The standards (and any recommended changes) will 
be reviewed and discussed at the pre-evaluation conference. The finalized list (complete with agreed 
changes) will t>e used for the enrichment specialist's evaluatKDn(s) during the coming school year. It is 
important to note that every standard must be linked directly to a performance indicator which is also linked 
directly to a duty. 

By giving the enrichment specialists a key role in the development of the standards, feelings of trust, 
respect, confidence, and fairness are cultivated and maintained. The program coordinator's involvement will 
ensure that attempts are made to note and incorporate what effective predecessors or personnel in similar 
organizations have done. The building administrator and program coordinator will be able to work together to 
see that contextual considerations (i.e., based on the needs of the program or school, the money, the 
facilities, and the time available) are honored by the standards set (Stronge & Helm, 1991). 

Stronge and Helm (1 991 , p 1 52) stated that "(a] level of performance is acceptable when it 
satisfactorily aids in fulfilling the job responsibility, thereby serving the needs of the institution and the clients 
(students) to be served " It is virtually impossible to define acceptable as it involves the judgments of the 
evaluator(s) and evaluatee and the context of the job performance at any given point in time. However, there 
IS no disputing the fact that some duties are more critical than others and that some employees have more 
critical job functions than do other employees. For these cntical functions and positions, there must be more 
stringent standards than might be set for less critical aspects of a given position. Factors to t>e considered in 
regards to this include: health and safety for students, educational significance, and cumulative effect 
(Stronge & Helm, 1991). 

Schoolwide enrichment specialists would have some critical duties that relate to these factors. For 
instance, when working with a group of students, it is essential that the specialist provide adequate 
supervision at all times to guarantee that the health and safety of the students are maintained. The 
educational significance of not following-up on a teacher recommendation of a student for identification into 
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the talent pool may mean that a student who should be receiving services is not. Some performance 
indicators of enrichment specialists alone may not seem critical; however, the combined impact of several 
indicators may be critical and provide a cumulative effect. One duty of an enrichment specialist is to provide 
information about the unique needs and characteristics of high ability learners for parents. Two performance 
indicators are identified: (a) features informational articles in regular newsletters; and (b) prepares and 
disseminates information for parents. If one of these performance indicators is done with little care or attention 
(if at all); then an important person in the life of a child may not have some very helpful information or may 
receive distorted information. 

it is imperative during this step of the evaluation process that everyone concerned be aware of critical 
duties and set high standards for the performance of these responsibilities. This information can also be 
valuable in the review of the job description and the duties. Nothing is more frustrating for a professional than 
to know that he or she has many duties, even critical ones, that do not receive proper attention t>ecause the 
context of the job (i.e., the time and monies allotted) is not adequate. This step requires such a close 
examination of the duties and a careful consideration of the performance indicators that it may inspire the 
revision of those duties and performance indicators based on the feasibility and reality of any one specialist 
being able to perform all the duties at a satisfactory (or t>etter yet, excellent) level. 

Some Thoughts on Step 4: Setting Performance Standards 

The concept of setting standards for measuring specific performance indicators is new, or at least 
foreign, to most educators. The initial involvement in this aspect of the evaluation process may cause 
fnjstration for evaluatees (and evaluators) due to a lack of skill and understanding. Yet, the step (and the 
complete collaboration between evaluatee and evaluator) is so vital to effective evaluation that proper training 
and resources must be made available to those involved. 

For the enrichment specialists, the program coordinator, and all the building administrators in the WJC 
Schools, there is a real need for professional development and training in writing appropriate standards. The 
handbook" Evaluating Professional Support Personnel in Education by Stronge and Helm (1 991 } has a 
chapter that more thoroughly describes the process than does this paper. However, having an expert in 
wnting evaluation standards provide inservice and feedback as each specialist/administrator team prepares the 
first list of standards would have the greatest impact on guai-anteeing the effective implementation of this 
evaluation model Every necessary arrangement should also be made to ensure that newly hired specialists 
receive the same training. 
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Document Job Performance 



In order to truly evaluate or to make any personnel decisions about an employee, there must be 
recorded evidence of the quantity and quality of his or her job performance - documentation must exist In 
order to truly interpret the level or degree of the quantity and/or quality of an employee's performance, this 
documentation must be composed of both pnmary and secondary data derived from a variety of sources 
(Stronge & Helm, 1991). This is especially true for professional support personnel whose job responsibilities 
are many and varied (if not unusual) and are not easily documented using standard (or at least commonly 
practiced) methods. As a matter of fact, many would agree that every professional deserves a more complete 
and diversified system of data collection and performance documentation than is currently utilized with most 
educators. 

This step in the evaluation process proposed by Stronge and Helm has the most potential for shifting 
the paradigm of personnel evaluation In many situations observation has become synonymous with 
evaluation and little regard is given to the many, varied, and unusual responsibilities of some staff members. 
While It IS true that the evaluation process should begin with the identification of system needs and progress 
through the steps of relating program expectations to duties, selecting performance indicators, and then 
setting standards. This may be an unrealistic ideal to think that many systems or programs will engage in such 
a process without a little more motivation than, "It should be done ' However, if educational organizations 
(systems, programs, or at least schools) will broaden their concepts of evaluation and data collection, then a 
realization of the vanous facets of an employee's job will become evident. This would then lead to a 
consideration of those duties (and performance indicators) and their relationship to the program's goals and 
the system s mission. Changing the methods for data collection may not be the first step in the process, but if 
rt IS all a school or system is willing or able to do at this time, it is, at least, a step in the right direction. 

As one reviews the list of the duties of a schoolwide ennchment specialist, it is obvious that many 
aspects of this job are not readily or easily observable Yet. the only method of evaluation used in the WJC 
Schools IS observation, though there is token attention given to the .'.etting of personal professional goals 
Not only is observation the only method of data collection, the instrument used for formal observations is a 

12 



ERIC 



checklist designed to reflect the major duties and prescribed stylistic processes of a classroom teacher. 

As was stated in the introduction, the WjC School District does recognize the diverse needs of its 
student population and, thus, employs many, different professional support personnel: guidance 
counselors, nurses, social workers, psychologists, media specialists, etc. However, most of these 
professionals are not evaluated using instruments and procedures that reflect their duties and none of these 
staff members, nor the programs they represent, have adopted a formal evaluation model like the one outlined 
by Stronge and Helm (1991). 

The VISIONS/Schoolwide Enrichment Program has been going through a rapid evolution, if not a 
metamorphosis, for the past four years. The Schootwide Ennchment Model developed by Joseph Renzulli 
has been adopted systemwide. Each irxiividual school under the leadership of the specialist who serves it 
has been modifying and refining this model so that it fits the needs and characteristics of the school. At this 
point in time, most of the schools have a working version of the model fully in place. Now it is time to build the 
supports that will guarantee that the program will contnue to improve. Some of these supports include 
personnel and program evaluation procedures and instruments. It could be argued that had such practices as 
examining the mission statement and developing program goals in keeping with this mission statement, 
reviewing the existing program, conducting a needs assessment, and planning for the evaluation of 
personnel been in place, the adoption of the Schoolwide Enrichment Program would have been better 
facilitated. 

The vanety of sources of documentation recommended by Stronge and Helm will allow for a complete 
evaluation of the many, different duties of the enrichment specialists and subsequent performance indicators. 
Because enrichment specialists work with students in minicourses and ennchment clusters, there is still a valid 
reason for using observation as a data gathering tool. For these observations, it would even be acceptable to 
use the checklist designed for classroom teachers, not because it is a valid instalment (we will leave the 
refinement and modification of this instrument for another time), but because in this capacity, we are acting as 
teachers. Observations could also be used to evaluate the quality of performance when a specialist conducts 
a staff development session, a teacher team meeting, or a parent workshop. 

Self-assessments would prove very enlightening for the ennchment specialist for many reasons. One 
way in which tne enrichment specialist would benefit is that an honest self-appraisal would reveal his or her 
progress toward achieving the individual professional goals that have been set for the year. This information 
would serve as a guideline in modifying behaviors or practices so that goals might be fully realized at the end of 
the year. The specialist would also be able to identify his or her strengths and weaknesses and alter his or her 
performance accordingly. The strongest argument, however, for using self -assessment as a means of 
evaluation is that it is a practice that is required of the primary constituents of these specialists ~ the gifted 
learners 

The duties of enrichment specialists clearly indicate that several constituencies exist for these PSP. It 
is only fitting that these groups be able to contribute to the evaluation of the specialists through surveys 
(questionnaires or interviews). Because building administrators and program coordinators, too, have diverse 
responsibilities, it is unlikely that either will be able to constantly monitor the effectiveness of a given 
employee. They must rely on the input of others who have more direct and constant contact with the 
employee, in this case, the other professional staff members, the students, and parents. Of these groups, a 
random sampling of those most impacted by the work of the specialist would be in order. 
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Artifact ccllection is absolutely necessary when evaluating enrichment specialists. Because of the 
nature of several duties, documents and records of all sorts are being generated by these PSP. Some of the 
artifacts generated by a resource specialist include monthly calendars, Type 1 announcements, minicourse 
syllabi and lesson plans, confidential data files on talent pool students, learning centers, catalogs, resource 
files, etc. If for no other reason than to acknowledge that the development of every artifact takes professionai 
time, talent, and effort, samples should be collected and reviewed. Of course, the artifacts reveal a great deal 
of infomnation about the spenalist and provide a superb source of low-inference data to achieve a well- 
balanced evaluation. An Artifact Analysis Form is provided in the appendix and represents one possible 
means of evaluating artifacts that are generated by the resource specialist throughout the year 

As this model is refined and adopted for use in evaluating the Sch'. olwide Enrichment Program's 
PSP, much professional time and energy must be dedicated initially to developing appropriate data collection 
instruments. That is, each enrichment resource specialist will need to collaborate with the program coordinator 
and txjilding administrator (the VISIONS staff might also choose to work together) to create observation forms, 
self-assessment report forms, and constituency questionnaires. Once these forms are developed, like the 
duties and performance indicators, they should only require minor modifications (if any) from year to year. 




At this stage of the evaluation process (or cycle), the evaiuator is trying to determine if there is any 
significant discrepancy between the achievements or behaviors of the employee (through documented job 
performance) and the predetermined standards of his or her duties (performance indicators). Once tnis 
determination is made, the matter then becomes one of how to use the information. Depending upon the 
point in the cycle at which the measurement and comparison occurs, the evaluation can be classified as 
formative or summative. Good personnel evaluations should have both fomnative and summative 
components 

The Evaluation Instruments and Performance Profiles 

Like any other tools and instruments, those developed for evaluations are useless without the 
intelligence and thoughtfulness of the evaiuator and evaluatee applied to them. The design of the evaluation 
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forms and p»ertormance analysis instalments is meant to convey information in the most helpful and concise 
manner; however, they are in no way meant to take the place of op»en, honest vertjal communication As a 
matter of fact, direct dialog must accompany the presentation of any fonms to mal<e them the meaningful, 
productivs agents for growth and development that they are meant to be. In the coming year, the VISIONS 
staff will need to work together to develop prototype instalments and a rating system for data collection and 
noting the level of performance for chosen standards. 

It should also be noted that, like any facet of the evaluation process, the fomis can and shouW be 
modified as the need to do so becomes apparent. Any changes that are incorporated should be discussed 
and decided upon by everyone involved -- the enrichment specialists, the program coordinator, and the 
building administrators. Just as there are infinite possibilities to design improvements for a mouse trap, there 
IS always room for making the instruments better fit the needs of the organization or school. 

The generation of perfonmance profiles (histograms) for each PSP after the summative evaluation at 
the end of the evaluation period is one way of graphically illustrating the overall performance rating. Profiles 
will be designed to show the specialist's performance in one area by charting the effectiveness rating of each 
indicator. Another profile can be generated to show all the areas which will reflect the specialist s overall 
performance. These profiles will be designed to give the specialist a visual interpretation of his or her 
performance and to aid in the development of professional development plan and performance standards 
proposal in the next evaluation period. 

improve/Maintain Professional Services 




Throughout this paper, this method of evaluating professional support personnel has been referred 
to as a process Perhaps a better term would be cycle. Indeed, this linal" step in the process is really the 
beginning of the next evaluation cycle. 

In fact, once the measurement has been conducted and any discrepancy between performance and 
objectives has been identified, a thorough evaluation system requires - especially in formative 
evaluation -- that the reason(s) behind the discrepancy be determined by an analysis of the context, 
input, and process, as well as the outcome. The analysis then can tie used as the basis for a variety of 
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actions ar.d decisions: diagnosis of problems with specific job performance; staff decisions; funding 
decisions: clarification of goals, philosophies, or standards; or as data in any accountability study 
(Stronge & Helm, 1991. p. 202). 

This paper is but a first step toward the ultimate goal of adopting and implementng a process that is 
appropriate for the evaluation of professional support personnel in the Williamsburg/James City County Public 
School System. There is much more to be done. Thus far the VISIONS staff has adopted the interpretation of 
the WJC mission statement and has even submitted this document to the state board of education as the 
"official' program goal statement. The staff members have also drafted and revised a list of duties. The next 
step in further developing this proposal for enrichment resource specialists is to have all the specialists and 
program coordinator submit possible indicators and standards. These will be compiled to create menus for use 
in the 1995-1996 school year when hopefully the evaluation system will be fully developed and instituted. 
Data gathering instruments that honor the job responsibilities and the characteristics and competencies listed 
above also need to be developed (with the input of building administrators where necessary). 

While there is much to be done in order to put this evaluation process in place and while the process 
itself will require that more time and energy be applied to the annual evaluation process, the benefits of 
instituting this process will be a stronger program staffed by competent professional who are wori<ing towards 
excellence in education. 
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Professional Support Personnel 
Evaluation Model 






step 7: 
Improve/Maintain 
Professional 
Service 



Step 1 : 
Identify 
System 
Needs 




Step 6: 
Evaluate 
Performance 



Systematic 
Communication 



Step 2: 
Identify 
Duties 



Step 5: 
Document Job 
Performance 



Step 4: 
Set 

Performance 
Standards 

'Utility .Accuracy 



Step 3: 
Select 
Performance 
Indicators 



gure 1 PSP Evaluation Model 



UJJC Mission Statement 



The mission of the Williamsburg/James City Ck)unty School Division Is to achieve 
excellence in education through partnership with home and community, so that 
each student beconnes: a llfe-iong learner, independent thinker and responsible 
citizen 



Ptnm m SUkKi in the 
WJC MtMion SUIenwnt 


IntaipraUiban of Ptraee ty the 
SchooMricie Enridwnant Program 


• v.. to achieve excellence in education...* 


• to promote the tadiation ai excellence* in both 
teachers and students 

• to foster a schoolwide atmosphere of excellence 


• "... through partnership with home and 

f^ommiinitv * 


• to impiement ways of utilizing parents and 
community members meaningfully in the pursuit 
of excellence in education 

• to encourage students and staff to use the 
community as a classroom without walls 


• "... each student..." 


• to provide opportunities for the development of 
gifted behaviors in all students 

* to assume that students of all ability levels benefit 
when the values of the school reflect an 
atmospiiere of excellence 


• "... a lite-long learner..." 


• to find and promote ways to honor students' 
strengths (above average abilities in any area), 
talents (in academic and nonacademic areas), 
interests (in all areas), learning styles, creativity, 
and affective needs to enhance their learning and 
desires to continue to learn 

• to integrate learning how-to-learn skills into the 
regular cumculum and other Type II training 
situations 


• ".. independent thinker..." 


• to integrate higher order thinking, creative 
thinking, and critical thinking skills into tfie regular 
curriculum and other Type 11 training situations 

• to facilitate independent investigatons/research 


• "... responsible citizen ' 


• to integrate affective thinking skills into the regular 
curriculum and other Type II training situations 

• to provide opportunities for students to t)ecomo 
creative producers and practicing professionals 

• to encourage the use of authentic assessment 
and accountability methods 



Figure 2 Developing Program Goals from the WJC Mission Statement 
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Schooiwide Enrichment Program 
Review Form 

School 

Level_ 

District Date 



Person Completing Form Title 



(1) rTOyrarn-spBCiTic inTorniaiion. 




(a) Review of present services 




• What is the description of services that have been 
established? 


• What are the tasks performed by the staff? 




• Does congruence exist tjetween the program 
description and actual services provided? 




• Are there unwritten expectations of the program'' If 
so. what are they? 




• Does the program's actual service delivery 
contribute to the accomplishment of the 
scnool s/ievei s/sysiems mission r 




(b) Community relations implications 




vvriai die ine pciocpituiio UI uic pruyicuit aiiu oiaii 

among internal audiences (e.g., students, staff)"? 


■• What i*? the Duhiic imaae of the Droaram"? 




• 1^ thp nrnnrflm r>prf:Pi\/P4d ac hinh niialrtv/ pnH 

lo iii^ L^i^Micini ^^71 v^Aj ' ' ' vfucii 1 1 y cii lu 

valuable for its intended constituents? 




(c) Status of resources committed to tiie orooram 




• What IS the operating budget"? 


• 1^ thp mnnpv hpinn PYnpnripri in mmnliJinpp with 

established objectives? 




• Is the current level of funding adequate? 




• What materials (e g , assessment instruments, 
professional puWications) are available"? 




• Are the types and numt)ers of professional and support staff 
adequate for desirsd program deTivery? 




• Are facilities appropriate"? 




(2) Position-specific information 


Please provide complete explanations below: 


(a) Review of position description 

• Are current job descnptions available for all staff 
members? 




• Are the job descriptions consistent with the 
demands of the program? 





er|c 



Program Review Form Continued 



1 

• Are job descriptions, taken individually, adequately 
defined to allow for the best use of personnel 
expertise'^ 




• Do the job descnpttons. taken as a whole, 
flryirnnnatplv rftlect the oroaram s obiectives and 
ttie schoors/level's/system's mission'^ 




(b) Analysis of time snd effort 




• Is documentation available regarding what staff 
rrrembers do and how they spend their time? 


• Does the enactment of posrtions match the job 
descriptions? 




• Do staff members invest their time wisely ? 




• Does a balanced work load exist among the staff 
members? 




• Are there duties and responsibilities that should be 
increased, reduced, eliminated? 




• Are there duties and responsibilities that should be 
transferred to other staff members or other 
programs? 


• 


(c) Asstsement of posttion-hokiers' competencies 




• What specific expertise exists among the staff? 


• Does assignment of actual duties and 
responsibilities match competencies of the staff 




• Do the competencies reflect the needed areas of 
program emphasis'? 




• What staff development efforts are needed to 
enhance the performance of staff 




(3) Client-specific Information 


Please provide complete explanations below 


(a) Review of student demographics 




• What IS an accurate descnption of the dient 
population (e.g., age. gender, race, socioeconomic 
factors)? 


♦ What IS the current enrollment or client population 
base (e g , by grade level, academic program type)? 




• What is the nature of the community from which the 
client population is drawn? 




• Are there additional clarifying charactenstics that 
reflect the background of the community in which 
the program operates? 
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Program Review Form Continued 



(b) Current us* of the proararn 




• Are accurate rocords available from which 
descriptive data of current use may be obtained? 


• What are the patterns of use (e g , time of day 
demands, heavy traffic areas within the program 
facilities)? 




• Which client subgroups make higher use of 
programs? 




• Which client sutjgroups make low demands? 




• Are adjustments in program services suggested by 
use patterns? 




Additional comments: 



I 



Schoolwide Enrichment Resource Specialist: 
Job Areas and Duties 



AREA: Planning & Intervention Services planned, developed, and/or delivered by 

the resource specialist directly to students. 

Duties: 

Plans Type I, Type 11, and Type 111 experiences for interested talent pool students (other 
students as needed). 

Provides (performs or facilitates) individual and group programming services and/or 
ennchment experiences (Type Type II, and Type 111) based on ability levei(s), talents, 
interests of student(s), existing curriculum guidelines, etc. 

Provides, when necessary, academic or affective counseling to talent pool students. 
Provides talent pool students with information about talent pool membership, program 
offerings, and extra-school opportunities. 

AREA: Staff Development 
Duties: 

Determines staff development needs for working with high ability learners, utilizing the 
Schoolwide Enrichment Program, or other topics traditionally associated with 
giftedrtalented education. 

Assists in the planning and implementation of training programs, workshops, or seminars, 
for the development and improvement of staff in areas related to gifted education. 
Aids in the onentation of new personnel. 

AREA: Liaison/Coordination 
Duties: 

Utilizes and coordinates community resources to serve student strengths, talents, 
interests, and needs. 

Provides public relations activities to communicate the program's goals and 
accomplishments (at twth distnct and school levels) to students, staff, parents, and 
community members 

Maintains communication with all persons involved .in or affected by any aspect of the 
Schoolwide Ennchment Program. 

AREA: Administration 
Duties: 

Organizes time, resources, and work load in order to meet responsibilities and allow for 
appropriate allocations to student services, staff services, parentycommunity services, 
and program administration. 

Implements and coordinates the Schoolwide Enrichment Program services and activities 
at the school level. 

Maintains accurate, complete, and punctual records as required by law, distnct policy, 
program or administrative directives, and the annual plan. 

Prepares budget recommendations and requests according to budgetary procedures 
and written guidelines. 

Prepares supplies and materials orders in keeping with budgetary parameters and written 
guidelines. 

Provides leadership of the ongoing program and in implementing new initiatives or 
changes. 

Assists in the defining and dissemination of information pertaining to program policies, 
procedures, and oftenngs 

Provides for use of current technologies available in program delivery and administration. 
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Mantains an organized, functional, accessible, and current resource room or center. 
Provides supervision of students in the resource room or in any activity sponsored by the 
enrichment specialist. 

AREA: Assessment/Evaluation 
Duties: 

Assesses ^tudents' interests (using appropriate instnjments)as a basis for planning and 
implement .,ig Type I and Type II activities. 

Uses intomiation gleaned from various levels of program review and needs assessments 
to make revisions and improvements in existing program. 
Assists in defining and reviewing criteria for talent pool placement and evaluations. 
Compiles, analyzes, and interprets data tor screening and identification of students for 
admission into or termination from the talent pool. 

Assists, as requested, in the evaluation of schoolwide and/or systemwide plans, policies, 
programs, etc. 

AREA: Professional Responsibilities/Development 
Duties: 

Adheres to ethical standards of the teaching profession. 

Adheres to guidelines of excellent teaching (as determined by WJC Schools) when 
working with students in a "class" setting. 

Observes policies and regulations in the delivery of sen/ices and management of the 
program. 

Participates in opportunities for continued personal and professional growth and 
» development. 

Plans and implements strategies for the attainment of personal professional goals and 
objectives. 

Assesses attainment of personal professional goals and objectives as described on the 
Professional Development Plan (PDP). 
Serves on the G/T Advisory Board. 

Serves on other committees, councils, or t)oards as appointed, assigned, or elected. 

AREA: Consultation 
Duties: 

Advocates for honoring the ability levels, talents, and interests of all students. 

Provides information atx)Ut the unique needs and characteristics of high ability learners to 

parents. 

Shares information and insights about individual students with school personnel, parents, 
and social agencies in a professional and confidential manner. 
Consults with students and parents to assist in meeting students' academic and/or 
affective needs. 

Assists regular classroom teachers and related arts teachers with planning and 
implementation of curncular initiatives as requested. 

Provides input, as requested, into schoolwide and/or systemwide policy and planning 

decisions as well as the development of the annual plan. 

Makes recommendations to individual teachers or teams of teachers: 

to integrate curncular and/or instructional differentiation techniques: 

to accommodate the unique ability levels, talents, interests, affective needs. 

and/or learning styles of specific students: 

to incorporate Type 1 and Type II expenences in the regular classroom. 
Makes recommendations for revisions to Schoolwide Enrichment Program or program 
offerings based on assessments and other data. 
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Schoolwide Enrichment Program 
Checklist for Developing Appropriate Standards 



Job Responsibility 

Performance Indicator 

Proposed Standard , 

This standard is designed to measure: (a) the outcomes of the professional performance; 

(b) the level of that performance as measured by the 
predetermined standards; 
O R (c) both the level of performance and the outcomes of 
that performance. 



Use the following crheria to decide if a proposed standard is appropriate. Rate each Item 
using this scale: 5 4 3 2 1 or N/A. 



Legal Considerations - 




Is the standard: 




• clear, meaningful, and fully comprehendible'? 




• )ustrfial)ie (i.e.. absolutely related to the employee s job performance 
or effectiveness)'? 




• feasonable (i.e., could the performance be compared to another 
emptoyee wrth similar responsit>ilities)? 




• communicated to the employee pnor to implementation/evaluation? 




• equitable and equitably implemented (nondiscriminatory) ? 




Evaluating Standards - 




uocs ine sianuaro nonor. 




• propnety standards'? 




• utility standards? 




• feasibtlitv •rfanriarrii'? 




• accuracy Standards'? 




Creating Standards • 




Have these considerations been applied to the standard 




• forniat (Is the standard stated as a behavioreil objective [observable and measurable 
with indications ot time and accuracy requirements when necessary]?); 




• quantity component (Where applicable, is an attempt made to identify how many, 
or ho'.v often [if at alO the performance indicators are exhibited?); 




• quality component (Is an attempt made to measure whether or not a behavior 
had a worthwhile impact or to measure the degree of quality of an activity?); 




• time component (Does the standard attempt to assess the amount and 
appropriateness of time spent on the indicator?); 




• cost component (Is there an attempt to assess the cost-effectiveness of a given 
performance indicator or at the relationship between funding and effectiveness?); 




• realistic expectations (Does this standard honor the mission, goals, and objectives 
of the system, the programn, and the individual? Does this standard reflect some 
degree of challenge without being unrealistic or meaningless?). 
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Artifact Analysis Form 



Complete this form for each artifact submitted by indicating the degree to which each criterion 
has been met. Assign ratings using ttie following scale: (3) = Excellent, 
(2) = Satisfactory, (1 ) = Unsatisfactory. The blank column can be used to indicate if the 
criterion is nonapplicable (N/A) or If a rating less than (1) is to be assigned. Attach a copy of 
the artifact being reviewed to the completed analysis form. 



Criteria 


3 


2 


1 




Comments 


Contant 












Validity: Is the artifact materially 
accurate arid authoritative'? 












Appropriateness: Is the content 
appropriate tor the task? 












Relevance: is the content relevant 
to the task? 












Motivation' Does the content motivate 
the receiver/user to cooperate or parti- 
apate' Does it stimulate interest"? 












Clarity Is the content tree of words, 
expressions, and graphics that would 
limit its comprehension? 












Ckinciseness Is the artifact free of 
superfluous words and matenaP Does 
it seek only relevant information'? 












Design 












Medium selection Is the most appro- 
pnate medium usedlor meeting the 
ob)ective(s) of the artrfacf? 












Meaningfulness Does the artifact clearly 
relate to the objective for which rt was de- 
signed"? 












Appropnateness: Is the design appropriate 
to the needs and interests of the receivers? 
Are time constraints considered in the design"? 












Sequencing: Is the artifact itself sequenced 
logicalty? Is it employed at the appropriate 
point in the presentation? 












Engagement: Does the artifact actively engage 
the receive/user? Does the appearance invrte 
the receiver to read or use it with interest? 












Evaluation. Is there a plan for evaluating the 
artifact's effectiveness? 
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Presentation 








- 




Effective use of time Is the artitact suitable 
lor ine ume aiiouea ' is ine rBceiver/user s 
time wasted by extraneous information or 
wordings? 












Aids to understandina Are directions cleariv 
explained? Are unfamiliar tenns defined? 
Are important concepts emphasized? 












Visual quality: Does the artifact present all 
the significant details? Is composition un- 
cluttered? Is ttiere aoDroDriate wie of hiah- 
lighting, color, c»ntrasts, space, pattern, 
arxj so forth? 












Phvsical aualitv' Is the artitact aonroDriateiv 
durable, attractive, and ample? Are size and 

^hanp nnnvpniPnt fnr otnranp and fiitiirp iiqa? 












CuhtQt A t& * 

«^ U W \ W i CI 1 o • 












Total Rating: 






Additional comments: 



Artifact submitted by: Date of submission: 



Artifact analyzed by: Date of analysis: 
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